Method for determining distance information from images of a spatial region

ABSTRACT

A method includes defining a disparity range having discrete disparities and taking first, second, and third images of a spatial region using first, second, and third imaging units. The imaging units are arranged in an isosceles triangle geometry. The method includes determining first similarity values for a pixel of the first image for all the discrete disparities along a first epipolar line associated with the pixel in the second image. The method includes determining second similarity values for the pixel for all discrete disparities along a second epipolar line associated with the pixel in the third image. The method includes combining the first and second similarity values and determining a common disparity based on the combined similarity values. The method includes determining a distance to a point within the spatial region for the pixel from the common disparity and the isosceles triangle geometry.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/EP2018/085562 filed on Dec. 18, 2018, which claims priority toGerman Patent Application No. 10 2017 130 897.0 filed on Dec. 21, 2017.The entire disclosures of the applications referenced above areincorporated by reference.

FIELD

The present disclosure relates to a method for determining distanceinformation from images of a spatial region and a correspondingapparatus. Furthermore, the disclosure relates to the use of suchapparatus for safeguarding a hazardous area of a technical installation.

BACKGROUND

Distance information of objects in a spatial region can be determined byoffset images of the spatial region. The offset (disparity) between theprojections of an object in the offset images depends on the distance ofthe object to the imaging units, so that with a known offset thedistance to an object can be determined by triangulation using theimaging geometry.

To determine the offset, corresponding elements must be found in theoffset images. This process is called correspondence analysis. Whilefinding corresponding elements in different images is easy for humans,it is a great challenge for a computer. Therefore, assignment errors canoccur, which ultimately lead to incorrectly measured distances.

In standard stereo systems, i.e. systems in which a stereo image takenwith two sensors is used, the assignment problems between the left imageand right images are countered with complex algorithms forcorrespondence analysis. Despite all efforts, however, assignmentproblems cannot be completely excluded or reliably reduced to atolerable level. It is therefore necessary, especially forsafety-critical applications where a distance must be reliably detectedat least in a defined range, to take additional measures to verify themeasured distance.

One way to improve the assignment quality is to add a further sensor toform another stereo pair to perform a complementary distancemeasurement. The determined distance values of both stereo pairs arethen used to check the results of the other stereo pair respectively. Inthis way, an attempt is made to reduce the overall assignment problems.If the determined distance values are equal, it can be assumed that thedistance is correct. However, if the results differ, a complexdecision-making process must either determine the priority of one of theresults or rejection of both results. Both require post-processing ofthe results, which significantly increases the complexity of theevaluation software.

SUMMARY

It is an object of the present disclosure to provide a method and acorresponding apparatus that enable an accurate disparity search.Further, it is an object to provide a method a corresponding apparatusthat can be implemented easily and efficiently. Furthermore, it is anobject to provide a method and a corresponding apparatus that do notrequire complex post-processing.

The object is solved by a method for determining distance informationfrom images of a spatial region, comprising:

-   -   arranging a first imaging unit, a second imaging unit, and third        imaging unit in a defined imaging geometry, in which the imaging        units form an isosceles triangle;    -   defining a disparity range having a number of discrete        disparities;    -   taking images of the spatial region, wherein a first image is        taken with the first imaging unit, a second image is taken with        the second imaging unit, and a third image is taken with the        third imaging unit;    -   determining first similarity values for at least one pixel of        the first image for all discrete disparities in the defined        disparity range along a first epipolar line associated with the        pixel in the second image;    -   determining second similarity values for the at least one pixel        of the first image for all discrete disparities in the defined        disparity range along a second epipolar line associated with the        pixel in the third image;    -   combining the first similarity values with the second similarity        values;    -   determining a common disparity between the first image, the        second image, and the third image for the at least one pixel        based on the combined similarity values; and    -   determining a distance to a point within the spatial region for        the at least one pixel from the common disparity and the defined        acquisition geometry.

According to a further aspect of the present disclosure, the object issolved by an apparatus for determining distance information from animage of a spatial region, comprising a first imaging unit for taking afirst image of the spatial region, a second imaging unit for taking asecond image of the spatial region, and a third imaging unit for takinga third image of the spatial region, wherein the first imaging unit, thesecond imaging unit, and the third imaging unit are arranged in adefined imaging geometry in which the imaging units form an isoscelestriangle, and an image processing unit configured to determine, withinthe defined disparity range having a number of discrete disparities,first similarity values and second similarity values for at least onepixel of the first image, wherein the image processing unit isconfigured to determine the first similarity values for the at least onepixel for all discrete disparities in the defined disparity range alonga first epipolar line in the second image associated with the pixel, anddetermine the second similarity values for the at least one pixel forall discrete disparities in the defined disparity range along a secondepipolar line in the third image associated with the pixel, wherein theimage processing unit is further configured to combine the firstsimilarity values with the second similarity values, determine a commondisparity for the at least one pixel between the first image, the secondimage, and the third image based on the combined similarity values, anddetermine a distance to a point within the spatial region for the atleast one pixel from the common disparity and the defined imaginggeometry.

It is thus an idea of the claimed solution to determine distanceinformation for a spatial region by means of three imaging units.However, instead of performing two stereo correspondence analyses fortwo feasible stereo pairs, an extended correspondence analysis isperformed, in which the similarity values from the image pairs inquestion are first combined and then jointly evaluated. Thus, not twoindividual distance measurements are performed, the results of whichbeing subsequently merged and compared, but only a single distancemeasurement, which, however, combines and equally considers theinformation of the imaging units.

This is possible due to a suitable epipolar geometry. The imaging unitsare arranged in a defined imaging geometry, in which two adjacentimaging units each generate an image pair in epipolar geometry and equalobject distances in all image pairs lead to the same disparities. Inparticular, the distance (base widths) between the first imaging unitand the second imaging unit, and the distance between the first imagingunit and the third imaging unit are equal, so that the imaging unitsform an isosceles (in special cases equilateral) triangle. The takenimages are then in a relationship to each other, which allows a simplelinking of the information from all imaging units. In other words, thecorrespondence analysis already makes use of extended information, whichresults from combining of the similarity values of the individual pairsof images.

As with known methods that use three imaging units, at least two datasets with similarity values are generated for two stereo image pairs.However, a subsequent disparity determination is not carried outseparately for each individual data set, but based on an aggregation ofboth data sets. Due to the suitable epipolar geometry resulting from thedefined imaging geometry, the aggregation can easily be achieved bymeans of a simple arithmetic operation.

Due to the aggregation of the similarity values before the actualcorrespondence analysis, an additional aggregation step and thus anadditional calculation step is necessary for the correspondenceanalysis. However, it has been shown that the additional aggregationstep can be implemented more easily than a complex post-processing.Since the aggregation is a simple operation that is applied equally toall similarity values, this additional calculation step can beimplemented with less computational effort than a post-processing of twoseparately determined distance information with a large number ofcomparison steps and decision steps. The implementation can thus besimplified overall.

The disclosed method thus enables a good assignment quality withmanageable level of complexity.

In a various embodiments, the imaging units can be arranged in anisosceles, right-angled triangle in the defined imaging geometry.

By the arrangement in a right-angled triangle, with one imaging unit atthe apex of the triangle and the other two imaging units at the end ofthe equal-length legs, the aggregation of similarity values can befurther simplified. An extended correspondence analysis can thus berealized easily, since the epipolar lines are perpendicular to eachother. In various embodiments, the epipolar lines may also beperpendicular to the respective image edges.

In a further refinement, the first similarity values are determined bycomparing the at least one pixel of the first image and its surroundingswith each pixel and its surroundings within the defined disparity rangealong the first epipolar line in the second image, and wherein thesecond similarity values are determined by comparing the at least onepixel of the first image and its surroundings with each pixel and itssurroundings within the defined disparity range along the secondepipolar line in the third image.

In this refinement, a local aggregation is performed to determine thesimilarity values. In this way, a reliable correspondence analysisbecomes possible, whereby the resulting similarity values can be easilycombined. For the comparison of the at least one pixel and itssurroundings with each pixel and its surroundings along the first andsecond epipolar line the sum of absolute differences or quadraticdifferences can be used.

In a further refinement, the first similarity values are added to thesecond similarity values.

In this refinement, the aggregation is achieved by simply adding thefirst and second similarity values. In other words, the similarityvalues determined in a defined disparity range for a defined number ofdiscrete disparities for a pixel of the first image with respect to thesecond image are each added to the values determined for the samedisparities for a pixel with respect to the third image. Performing andaddition is a simple arithmetic operation that can be implementedquickly and in a resource-saving manner. The refinement thus contributesfurther to an efficient implementation of the method.

In a further refinement, determining the common disparity for the atleast one pixel includes an extreme value search in the combinedsimilarity values, in particular a search for a minimum.

In this refinement, the disparity is determined by means of an extremevalue search within the combined similarity values. Thus, fordetermining the disparity form the combined disparity values the sameapproach can be used as for known stereo correspondence analyses. Theimplementation of the method is simplified, since existing algorithmscan be reused.

In a further refinement, the first image, the second image, and thethird image are transformed to each other so that the first epipolarline extends along a first axis and the second epipolar line extendsalong a second axis perpendicular to the first epipolar line.

In this refinement, the alignment of the images to each other in thedefined imaging geometry can be further optimized. In variousembodiments, the first image, the second image, and the third image eachcan have an equal number of pixel lines and an equal number of pixelcolumns, wherein the first epipolar line extends along a pixel line inthe second image corresponding to the pixel line of the first image inwhich the at least one pixel is located, and wherein the second epipolarline extends along a pixel column in the third image corresponding tothe pixel column of the first image in which the at least one pixel islocated. In this refinement, the images can be processed in rows andcolumns, which makes image processing simple and efficient.

In a further refinement, a common disparity is determined for all pixelsof the first image. Alternatively, a common disparity can be determinedfor only a defined number of pixels of the first image. While onerefinement allows the highest possible resolution of a depth map, theother refinement allows the depth value determination to be limited toonly relevant areas of an image, which allows a faster calculation ofonly relevant areas.

In a further refinement, the method further comprises determining thirdsimilarity values for at least one further pixel of the second image forall discrete disparities in a defined disparity range along a firstepipolar line in the first image associated with the further pixel,determining fourth similarity values for the at least one further pixelof the second image for all discrete disparities in the defineddisparity range along a second epipolar line in the third imageassociated with the further pixel, and determining further distanceinformation from the third and fourth similarity values.

In this refinement, at least one further distance information is thusdetermined by using one of the other two images as a reference image,wherein third and fourth similarity values are determined based on theselected reference image and used to determine a further distanceinformation. Determining the further distance information may alsoinvolve aggregation of the third similarity values and the fourthsimilarity values and determining another common disparity.Alternatively, the further distance information can be determined fromthe third similarity values and the fourth similarity values in aconventional way. Thereby, not only redundant distance information canbe determined, but also diversity of the process can be increased.Overall, the refinement allows verifying the determined distanceinformation in a simple manner. It is understood that a further increasein quality can be achieved by using each of the images of the first,second and third imaging unit as a reference image.

It is understood that the features mentioned above and the features tobe explained below can be used not only in the combination indicated,but also in other combinations or uniquely, without leaving the scope ofthis disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are shown in the drawings and are explainedin more detail in the following description.

FIG. 1 shows a schematic view of an apparatus according to an exampleembodiment.

FIG. 2 shows a perspective of an apparatus according to an exampleembodiment.

FIG. 3 shows a block diagram of a method according to an exampleembodiment.

FIG. 4 shows a schematic view of three images of the first imaging unit,the second imaging unit, and the third imaging unit.

FIG. 5 shows a diagram view of first similarity values and secondsimilarity values.

FIG. 6 shows a diagram view of combined similarity values.

FIG. 7 shows an application scenario of an apparatus and methodaccording to an example embodiment.

DETAILED DESCRIPTION

In FIG. 1, an apparatus according to an example embodiment is denoted inits entirety with reference numeral 10.

In this example embodiment, the apparatus 10 comprises three imagingunits 12, 14, 16, each of which is designed to take an image of aspatial region 18 laying in front of the units. The imaging units 12,14, 16 can be digital camera units with one optic each for projectingthe spatial region 18 onto an image sensor of the camera unit. Eachpoint in space in the spatial region 18 is uniquely assigned to a pixelof the image sensor. The imaging units 12, 14, 16 can be complete,independently functioning cameras, or a combined unit, in whichindividual parts of the imaging units are used together, so that theyessentially act as three separate image sensors with associated optics.

The three imaging units 12, 14, 16 are arranged in a defined imaginggeometry 20, which is indicated here by the dashed lines. In the definedimaging geometry 20, one of the imaging units 12 forms a central unitfrom which the other two imaging units 14, 16 are arranged at an equaldistance. The imaging units 12, 14, 16 thus form an isosceles triangle,with the imaging units forming the vertices of this triangle.

The imaging units 12, 14, 16 can form a right-angled, isoscelestriangle. The central imaging unit 12 sits at the apex and the other twoimaging units 14, 16 are located in the remaining corners of thetriangle. The legs of the triangle enclose an angle of 90°.

From the offset images of the spatial region 18 provided by the imagingunits 12, 14, 16, distance information of objects 22, 24 in the spatialregion 18 can be determined, since the offset between the projections ofthe objects 22, 24 in the offset images depends on the distance of theobjects 22, 24 to the imaging units 12, 14, 16. By means oftriangulation, the distance to objects 22, 24 can be determined from theknown offset using the imaging geometry 20. In addition to the distanceto an object 22, 24, the relative position of the objects 22, 24 to eachother can also be determined without further sensors.

The processing of the images of the imaging units 12, 14, 16 accordingto the example embodiment shown here is carried out by an imageprocessing unit 26, which determines the distance information to theindividual objects 22, 24, which are located in the spatial region 18,from the images. For this purpose, the image processing unit 26 forms atleast two image pairs from the images of the three imaging units 12, 14,16 and determines a set of similarity values for each pair. Due to thedefined arrangement of the imaging units 12, 14, 16, the similarityvalues of the image pairs can be directly combined with each other,which enables a joint evaluation of the similarity values, which isexplained in more detail below.

The joint evaluation of the similarity values enables an accuratedisparity determination, wherein time-consuming post-processing is notnecessary. The method is thus essentially a one-step process, using anextended correspondence analysis that can take into account theinformation from several sensors. Compared to a two-step process havingtwo distance determination processes and a subsequent plausibilitycheck, the software implementation can be simplified, which especiallyfacilitates porting the method to architectures designed for linearprocessing without many control structures, such as an FPGA.

In various embodiments, the image processing unit 26 can be a FPGA.Furthermore, the image processing unit 26 can be enclosed together withthe imaging units 12, 14, 16 in a common housing 28. It is understoodthat in other embodiments, the image processing unit 26 can also be aseparate unit outside the housing 28.

FIG. 2 shows an apparatus according to an example embodiment of thedisclosure in a perspective view, in particular the integration of thethree imaging units 12, 14, 16 and the image processing unit 26 in acommon housing 28. Same reference signs here designate the same parts aspreviously in FIG. 1.

The apparatus 10 is integrated in a housing 28 and mountable on a wall,a mast or the like via a mounting part 30. A base 32 of the apparatusfacing the monitoring spatial region (not shown here) has three openings34 a, 34 b, 34 c, behind which the three imaging units 12, 14, 16 arelocated inside the housing 28. The optics of the imaging units 12, 14,16 are located directly behind the central openings 36 a, 36 b, 36 c.

The connecting lines 38 through the openings 36 form an isoscelestriangle in the base 32, which represents the defined imaging geometry20. The distance from the first imaging unit 12 to the second imagingunit 14 is thus equal to the distance from the first imaging unit 12 tothe third imaging unit 16. As shown here, the defined imaging geometry20 can also be a right-angled, isosceles triangle, i.e. the angle formedby the straight lines at the opening 36 a is a 90° angle.

FIG. 3 shows a block diagram of a method according to an exampleembodiment. The method in its entirety is denote here with referencenumeral 100.

In a first step 102, a first, second and third imaging unit are arrangedin a defined imaging geometry in which the imaging units form anisosceles triangle. This step may also include a calibration of theimaging units.

In a next step 104, a disparity range with a number of discretedisparities is determined. The disparity range indicates the range ofpossible disparities between two pairs of images. The disparity rangecan be specified by a minimum and a maximum disparity, wherein thediscrete disparities are corresponding intermediate values with adefined distance in between. For example, if the minimum disparity iszero and the maximum disparity is ten pixels, the disparity range, at adefined distance of one pixel, includes all pixels between the minimumand the maximum disparity.

In step 106, the spatial region is imaged, wherein a first image istaken with the first imaging unit, a second image is taken with thesecond imaging unit, and a third image is taken with the third imagingunit. During imaging, each point in the spatial region is uniquelyassigned to a pixel of the image sensors of the first, second and thirdimaging unit. Accordingly, at a defined point in time, there are threeindividual offset images of the spatial region. The step may furtherinclude rectification of the images.

In step 108, first similarity values for a pixel of the first image aredetermined for all discrete disparities in the defined disparity rangealong a first epipolar line in the second image associated with thepixel. In this step, a comparison between the first and the second imageis made. The similarity of a pixel is determined with respect to thesame pixel in the second image and all other pixels starting from thesame pixel in the entire disparity range. Due to the epipolar geometry,a corresponding pixel can only appear along the epipolar line, so thatthe search is limited to this area. The first similarity values includefor a reference pixel in the first image for all discrete disparities asimilarity value that describes the similarity of the reference pixel tothe respective pixel in the second image at the distance of the discretedisparity.

In other words, at first, it is determined how similar the pixel of thefirst image is compared to the corresponding pixel of the second image.Then the similarity of the first pixel to the neighboring pixel of thecorresponding pixel is determined. This process is repeated untilsimilarity values for all discrete disparities up to the maximumpossible disparity have been determined.

When determining the similarity values, not only are the individualpixels compared to each other to determine their similarity, but theimmediate surroundings of the pixel are also taken into account.Therefore, a classic block-matching procedure may be applied, in whichthe sum of the absolute (SAD values) or quadratic (SSD values)differences are calculated for every possible disparity, for example in7×7 window sections, between the first image and the second image. Thesevalues are a measure of the similarity of the window sectionsconsidered. The more similar the window sections are, the lower thesimilarity value. The smallest possible similarity value in this case iszero and represents identical window sections.

In step 110, second similarity values are determined by applying theprocess described in step 108 to the remaining image pair. The secondsimilarity values are therefore also, for example, SAD or SSD valuescalculated for any disparity between the first image and the thirdimage. The second similarity values can be determined after the firstsimilarity values, or in parallel.

Once the first and second similarity values have been determined, theyare appropriately combined (aggregated) in step 112. This can beachieved by applying a simple arithmetic operation to the first andsecond similarity values of the same disparity. In various embodiments,the corresponding similarity values can be added together to obtain thecombined similarity values. It is understood that normally not integerdisparities are used, but subpixel disparities are used instead.

After combining the similarity values, the disparity is determined.Finding the disparity involves searching for a minimum in the combinedsimilarity values. Since there can be one or more smallest values, thesearch is not only for the smallest value, but for the point where thereis a clear minimum. To find such a position, for example, a thresholdvalue relative to the smallest value can be used. If the smallest valueis 1000 and the threshold 20%, all values below 1200 are determined. Twovalues below this threshold are also allowed, provided they are adjacentto each other. If this condition cannot be met, no assignment ispossible.

Since the similarity values of the first and second and the first andthird images should lead to the same disparities due to the definedimaging geometry at the same object distances, a common disparitybetween the first image, the second image, and the third image isconsequently determined in step 114 on the basis of the combinedsimilarity values for the at least one pixel.

Subsequently, in step 116, the distance is determined for the at leastone pixel based on the common disparity. Here, the distance iscalculated by triangulation from the known imaging geometry and thecommon disparity in a known manner.

It goes without saying that the above procedure cannot be applied to asingle pixel of the image, but that distance information can bedetermined for each pixel. Furthermore, it is conceivable that distanceinformation may only be determined for a defined pixel area of the firstimage in order to limit the evaluation to a relevant area.

Thus, steps 108 to 116 are repeated for each further relevant pixel orpixels of the first image until a corresponding distance information hasbeen determined for all relevant pixels.

It is understood that the procedure can include further preparation andfollow-up steps in addition to those described here. Likewise, the stepsdo not have to be executed sequentially. In various embodiments, somesteps, such as determining the first similarity values and the secondsimilarity values, can be executed in parallel. In addition, it isconceivable that not single images are taken, but continuous imagesequences are considered from which specific individual images areextracted.

FIG. 4 shows an example of a first image 40, a second image 42 and athird image 44. The first image 40 corresponds to an image taken withthe first imaging unit 12, the second image 42 corresponds to asimultaneous image taken with the second imaging unit 14, and the thirdimage 44 corresponds to another simultaneous image taken with the thirdimaging unit 16. All three images show an image of the same spatialregion in which an object is arranged, which is here, as an example,indicated by three slashes “///”.

Each image 40, 42, 44 corresponds to a pixel matrix with a multitude ofpixel lines 46 and a multitude of pixel columns 48. In the exampleembodiment shown here, the image sensors of the imaging units 12, 14, 16have the same resolution and identical pixel columns and pixel lines.The first image 40, the second image 42 and the third image 44 thus havean equal number of pixel lines 46 and pixel columns 48.

Each point of the spatial region is assigned to a pixel of the pixelmatrix and visible points of the spatial region are mapped on the pixelmatrix. Since the imaging units are offset to each other, the same pointof the spatial region on the first image 40, the second image 42 and thethird image 44 can be mapped to different pixels of the pixel matrix ofthe respective image.

Reference numerals 50 and 52 denote epipolar lines that result from thedefined imaging geometry. In addition to the physical alignment of thethree imaging units, further calibration steps may be necessary,especially rectification of the image to obtain the desired epipolargeometry. After the alignment, the first epipolar line 50 runsperpendicular to the second epipolar line 52.

By correspondence analysis, correspondences in the second image 42 andthe third image 44 for the pixels of the first image 40 are searched, inorder to determine subsequently the disparity of depicted objects in theimages. This is shown below using pixel 54 of the first image 40 as anexample.

Due to the epipolar geometry in the second image 42, a correspondence topixel 54 must lie on the epipolar line 50. Likewise, a correspondence topixel 54 in the third image 44 must be on the second epipolar line 52.Not all pixels along the epipolar line 50, 52 have to be considered, butonly those within a defined disparity range 56.

The disparity range 56 extends here from a pixel 58 corresponding topixel 54 in the second and third images along the first and secondepipolar lines 50, 52, respectively, and is limited by a maximumdisparity 60. The search for possible correspondences is limited to thedefined disparity range 56. In the example embodiment shown here, eachpixel within the disparity range 56 describes a possible discretedisparity. For the search for correspondences, starting from thecorresponding pixel 58 along the first or second epipolar line 50, 52,each pixel in the disparity range 56 to the maximum disparity 60 iscompared with the reference pixel 54. Not only the respective pixelsthemselves can be considered, but also their surroundings. In otherwords, a so-called block matching procedure can be applied.

From the example shown here, it can be seen that when comparing thefirst image 40 with the second image 42, three pixels 62 a, 62 b, 62 cwithin the disparity range 56 provide the same similarity measures forpixel 54. Therefore, the search for correspondence within the disparityrange 56 provides no clear result in this case.

The search for correspondences in the first image 40 and in the thirdimage 44, on the other hand, provides a clear result. For the pixel 54along the second epipolar line 52 in the third image 44, there is onlyone possible correspondence 64; in other words, while for pixel 54 acorrespondence analysis with the second image 42 would lead toassignment problems, a correspondence analysis between the first image40 and the third image 44 would provide a clear solution.

The method according to the disclosure suggests combining the similarityvalues determined in connection with the search for correspondences inthe first image 40, the second image 42 and the third image 44 beforethe similarity values are further evaluated. The aggregation between thesimilarity values is explained in more detail below using FIG. 5 andFIG. 6.

In FIG. 5, first similarity values 66 and second similarity values 68are plotted in a first diagram and a second diagram, which weredetermined for a pixel 54 of the first image 40 during the search forpossible correspondences in the second image 42 and the third image 44.In the upper diagram, the similarity values for a pixel 54 along thefirst epipolar line 50 are plotted and in the lower diagram, thesimilarity values for the same pixel 54 along the second epipolar line52 are plotted. In total, similarity values along the x-axis wereplotted here for thirty-two discrete disparities. The lower therespective value, the more similar the first pixel 54 and the consideredpixel are. According to the example in FIG. 4, the upper diagram showsthree minima. In other words, for three specific disparities, thesimilarity measures between the pixels under consideration are thesmallest. Therefore, it is not possible to determine the disparityclearly from these similarity values. In the lower diagram 68, on theother hand, there is exactly one minimum, i.e. for a concrete disparitythe similarity between the observed pixels is the smallest.

The method according to the disclosure proposes combining the firstsimilarity values 66 and the second similarity values 68 with eachother, so that the similarity values 66, 68 are already combined beforean extreme value search. An example of this is shown in FIG. 6.

FIG. 6 shows the combined similarity values 70 of the similarity valuesshown in FIG. 5. The aggregation is achieved by a simple addition of thefirst similarity values 66 and the second similarity values 68 that havebeen previously determined from the image pairs. In this way, extremevalues are highlighted more clearly, as there is a larger informationbase for the same information. As can be seen from FIG. 6, the combinedsimilarity values only have a clear minimum at one position. The methodaccording to the disclosure is thus able to determine clear and correctdistance information for sufficiently pronounced structures in the imagealmost independently of their orientation.

Finally, FIG. 7 shows an application scenario of an apparatus and amethod according to an example embodiment.

In FIG. 7, an apparatus 10 according to the disclosure is used toprotect a technical installation 72. The technical installation 72 isindicated here by an industrial robot 74. The robot 74 comprises amanipulator 76, which can move freely in space and whose movement rangedefines a hazardous area 78 of the technical installation 72. Within thehazardous area 78, the movement of the manipulator 76 may pose a dangerto an operator 80, so that when entering the hazardous area 78 of thetechnical installation 72, the manipulator 76 must be transferred into astate that is safe for the operator 80.

In this example embodiment, the apparatus is placed above the technicalinstallation 72, for instance on a ceiling of an assembly hall, so thatthe imaging units 12, 14, 16 can take images of the spatial regionaround the technical installation 72.

In the manner described above, the apparatus 10 can determine distanceinformation for the spatial region. Certain areas, such as here themovement area, can be excluded from the determination of the distanceinformation. As soon as an operator enters the spatial region around thetechnical installation 72, the distance information for this areadetermined by the apparatus 10 changes. By continuously determiningdistance information and registering changes, the apparatus 10 is ableto detect a person or object within the spatial region.

The apparatus 10 can thus signal the entry of a person into the spatialregion based on the distance information determined and, for example,transfer the technical installation into a safe state via a safety meansconnected to the apparatus 10 (not shown here). The safety means may,for example, be an ordinary safety-switching device or a safe controlsystem that are configured to ensure safety in a known manner, forexample by switching off the technical installation 72.

The method allows the apparatus, including the image processing device,to be integrated in a housing, so that the apparatus can directlyinteract with the safety means as a safe sensor. This is possiblebecause the method can be ported to hardware specialized for linearprocessing.

The term non-transitory computer-readable medium does not encompasstransitory electrical or electromagnetic signals propagating through amedium (such as on a carrier wave). Non-limiting examples of anon-transitory computer-readable medium are nonvolatile memory circuits(such as a flash memory circuit, an erasable programmable read-onlymemory circuit, or a mask read-only memory circuit), volatile memorycircuits (such as a static random access memory circuit or a dynamicrandom access memory circuit), magnetic storage media (such as an analogor digital magnetic tape or a hard disk drive), and optical storagemedia (such as a CD, a DVD, or a Blu-ray Disc).

The phrase at least one of A, B, and C should be construed to mean alogical (A OR B OR C), using a non-exclusive logical OR, and should notbe construed to mean “at least one of A, at least one of B, and at leastone of C.”

What is claimed is:
 1. A method for determining distance information,the method comprising: defining a disparity range having a number ofdiscrete disparities; taking a first image of a spatial region with afirst imaging unit, a second image of the spatial region with a secondimaging unit, and a third image of the spatial region with a thirdimaging unit, wherein the first imaging unit, the second imaging unit,and the third imaging unit are arranged in a defined imaging geometry inwhich the imaging units form an isosceles triangle; determining firstsimilarity values for at least one pixel of the first image for alldiscrete disparities in the defined disparity range along a firstepipolar line associated with the pixel in the second image; determiningsecond similarity values for the at least one pixel of the first imagefor all discrete disparities in the defined disparity range along asecond epipolar line associated with the pixel in the third image;combining the first similarity values with the second similarity values;determining a common disparity between the first image, the secondimage, and the third image for the at least one pixel based on thecombined similarity values; and determining a distance to a point withinthe spatial region for the at least one pixel from the common disparityand the defined imaging geometry.
 2. The method of claim 1, furthercomprising: carrying out a scene analysis to detect foreign objects in ahazardous area of a technical installation based on the distance to thepoint within the spatial region; and transferring the technicalinstallation into a safe state in response to detection of a foreignobject.
 3. The method of claim 1 further comprising: based on thedistance to the point within the spatial region, selectively detecting aforeign object within the spatial region; and in response to detectionof the foreign object, switching off a robot operating within thespatial region.
 4. The method of claim 1, wherein the first imagingunit, the second imaging unit, and the third imaging unit form anisosceles, right-angled triangle in the defined imaging geometry.
 5. Themethod of claim 1, wherein: the first similarity values are determinedby comparing the at least one pixel of the first image and itssurroundings with each pixel and its surroundings within the defineddisparity range along the first epipolar line in the second image, andthe second similarity values are determined by comparing the at leastone pixel of the first image and its surrounding with each pixel and itssurroundings within the defined disparity range along the secondepipolar line in the third image.
 6. The method of claim 5, furthercomprising determining, for comparison of the at least one pixel and itssurroundings with each pixel and its surroundings along the firstepipolar line and the second epipolar line, a sum of at least one ofabsolute differences and quadratic differences.
 7. The method of claim1, wherein the first similarity values and the second similarity valuesare added together.
 8. The method of claim 1, wherein determining thecommon disparity for the at least one pixel includes an extreme valuesearch in the combined similarity values.
 9. The method of claim 8,wherein the extreme value search is a search for a minimum.
 10. Themethod of claim 1, wherein the first image, the second image, and thethird image are transformed relative to each other such that the firstepipolar line extends along a first axis and the second epipolar lineextends along a second axis perpendicular to the first epipolar line.11. The method of claim 10, wherein: the first image, the second image,and the third image each comprise an equal number of pixel lines and anequal number of pixel columns, the first epipolar line extends in thesecond image along a pixel line that corresponds to the pixel line ofthe first image in which the at least one pixel is located, and thesecond epipolar line extends in the third image along a pixel columnthat corresponds to the pixel column of the first image in which the atleast one pixel is located.
 12. The method of claim 1, wherein a commondisparity is determined for all pixels of the first image.
 13. Themethod of claim 1, wherein a common disparity is determined for adefined number of pixels of the first image only.
 14. The method ofclaim 1, further comprising: determining third similarity values for atleast one further pixel of the second image for all discrete disparitiesin the defined disparity range along a first epipolar line in the firstimage associated with the further pixel; determining fourth similarityvalues for the at least one further pixel of the second image for alldiscrete disparities in the defined disparity range along a secondepipolar line associated with the further pixel in the third image; anddetermining further distance information from the third and fourthsimilarity values.
 15. An apparatus for determining distance informationfrom images of a spatial region, the apparatus comprising: a firstimaging unit configured to take a first image of the spatial region; asecond imaging unit configured to take a second image of the spatialregion; a third imaging unit configured to take a third image of thespatial region; and an image processing unit configured to determinefirst similarity values and second similarity values for at least onepixel of the first image within a defined disparity range having anumber of discrete disparities, wherein the first imaging unit, thesecond imaging unit, and the third imaging unit are arranged in adefined imaging geometry in which the imaging units form an isoscelestriangle, wherein the image processing unit is configured to: determinethe first similarity values for the at least one pixel for all discretedisparities in the defined disparity range along a first epipolar lineassociated with the pixel in the second image, and determine the secondsimilarity values for the at least one pixel for all discretedisparities in the defined disparity range along a second epipolar lineassociated with the pixel in the third image, and wherein the imageprocessing unit is further configured to: combine the first similarityvalues with the second similarity values, determine a common disparityfor the at least one pixel between the first image, the second image,and the third image based on the combined similarity values, anddetermine a distance to a point within the spatial region for the atleast one pixel from the common disparity and the defined imaginggeometry.
 16. The apparatus of claim 15, wherein the image processingunit is an FPGA.
 17. The apparatus of claim 15, further comprising: anevaluation unit configured to carry out a scene analysis to detectforeign objects in a hazardous area of a technical installation based onthe distance to the point within the spatial region; and safetyequipment configured to transfer the technical installation into a safestate in response to the evaluation unit detecting a foreign object. 18.The apparatus of claim 15, wherein the first imaging unit, the secondimaging unit, and the third imaging unit are arranged in a commonhousing.
 19. A non-transitory computer-readable medium comprisinginstructions including: arranging a first imaging unit, a second imagingunit, and a third imaging unit in a defined imaging geometry, in whichthe imaging units form an isosceles triangle; defining a disparity rangehaving a number of discrete disparities; taking a first image of aspatial region with the first imaging unit, a second image of thespatial region with the second imaging unit, and a third image of thespatial region with the third imaging unit; determining first similarityvalues for at least one pixel of the first image for all discretedisparities in the defined disparity range along a first epipolar lineassociated with the pixel in the second image; determining secondsimilarity values for the at least one pixel of the first image for alldiscrete disparities in the defined disparity range along a secondepipolar line associated with the pixel in the third image; combiningthe first similarity values with the second similarity values;determining a common disparity between the first image, the secondimage, and the third image for the at least one pixel based on thecombined similarity values; and determining a distance to a point withinthe spatial region for the at least one pixel from the common disparityand the defined imaging geometry.
 20. The computer-readable medium ofclaim 19 further comprising instructions including: based on thedistance to the point within the spatial region, selectively detecting aforeign object within the spatial region; and in response to detectionof the foreign object, switching off a robot operating within thespatial region.